DUC 2005: Evaluation Of Question-Focused Summarization Systems
نویسنده
چکیده
The Document Understanding Conference (DUC) 2005 evaluation had a single useroriented, question-focused summarization task, which was to synthesize from a set of 25-50 documents a well-organized, fluent answer to a complex question. The evaluation shows that the best summarization systems have difficulty extracting relevant sentences in response to complex questions (as opposed to representative sentences that might be appropriate to a generic summary). The relatively generous allowance of 250 words for each answer also reveals how difficult it is for current summarization systems to produce fluent text from multiple documents.
منابع مشابه
Fudan University at DUC 2005
In this paper, we described our participation in the question-focused multi-document summarization task of DUC 2005. Our system was based on a supervised machine learning method in which feature extraction was an important issue. We present the whole procedure of our system, focusing on the features we used. We also analyze the results of manual and automatic evaluations.
متن کاملFocused multi-document summarization: Human summarization activity vs. automated systems techniques
Focused Multi-Document Summarization (MDS) is concerned with summarizing documents in a collection with a concentration toward a particular external request (i.e. query, question, topic, etc.), or focus. Although the current state-of-the-art provides somewhat decent performance for DUC/TAC-like evaluations (i.e. government and news concerns), other considerations need to be explored. This paper...
متن کاملDescription of SQUASH, the SFU Question Answering Summary Handler for the DUC-2006 Summarization Task
This paper describes the design of the SQUASH system, the SFU Question Answering Summary Handler, developed by members of the Natural Language Lab from the SFU School of Computing Science in order to participate in the 2006 Document Understanding Conference (DUC-2006) summarization task. The SQUASH system for DUC-2006 built upon the SQUASH system that was built for the last year’s DUC-2005 comp...
متن کاملDescription of SQUASH, the SFU Question Answering Summary Handler for the DUC-2005 Summarization Task
This paper describes the design of the SQUASH system, the SFU Question Answering Summary Handler, developed by members of the Natural Language Lab from the SFU School of Computing Science in order to participate in the 2005 Document Understanding Conference (DUC-2005) summarization task. The system design involves semantic role labelling, semantic subgraph-based sentence selection and automatic...
متن کاملCLASSY Query-Based Multi-Document Summarization
Our summarizer is based on an HMM (Hidden Markov Model) for sentence selection within a document and a pivoted QR algorithm to generate a multi-document summary. Each year, since we began participating in DUC in 2001, we have modified the features used by the HMM and have added linguistic capabilities in order to improve the summaries we generate. Our system, called “CLASSY” (Clustering, Lingui...
متن کامل